Stochastic recursive inclusions with two timescales
نویسندگان
چکیده
In this paper we present a framework to analyze the asymptotic behavior of two timescale stochastic approximation algorithms including those with set-valued mean fields. This paper builds on the works of Borkar and Perkins & Leslie. The framework presented herein is more general as compared to the synchronous two timescale framework of Perkins & Leslie, however the assumptions involved are easily verifiable. As an application, we use this framework to analyze the two timescale stochastic approximation algorithm corresponding to the Lagrangian dual problem in optimization theory.
منابع مشابه
Stochastic Recursive Inclusions in two timescales with non-additive iterate dependent Markov noise
In this paper we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise. It is shown that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures...
متن کاملStochastic differential inclusions of semimonotone type in Hilbert spaces
In this paper, we study the existence of generalized solutions for the infinite dimensional nonlinear stochastic differential inclusions $dx(t) in F(t,x(t))dt +G(t,x(t))dW_t$ in which the multifunction $F$ is semimonotone and hemicontinuous and the operator-valued multifunction $G$ satisfies a Lipschitz condition. We define the It^{o} stochastic integral of operator set-valued stochastic pr...
متن کاملA Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions
In this paper the stability theorem of Borkar and Meyn is extended to include the case when the mean field is a set-valued map. Two different sets of sufficient conditions are presented that guarantee the ‘stability and convergence’ of stochastic recursive inclusions. Our work builds on the works of Benäım, Hofbauer and Sorin as well as Borkar and Meyn. As a corollary to one of the main theorem...
متن کاملA Stochastic Model for Water Resources Management
Irrigation water management is crucial for agricultural production and livelihood security in many regions and countries throughout the world. Over the past decades, controversial and conflictladen water-allocation issues among competing municipal, industrial and agricultural interests have raised increasing concerns. Particularly, growing population, varying natural conditions and shrinking wa...
متن کاملTwo Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning
We present for the first time an asymptotic convergence analysis of two-timescale stochastic approximation driven by controlled Markov noise. In particular, both the faster and slower recursions have non-additive Markov noise components in addition to martingale difference noise. We analyze the asymptotic behavior of our framework by relating it to limiting differential inclusions in both times...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1502.01956 شماره
صفحات -
تاریخ انتشار 2015